Reduction of Maximum Entropy Models to Hidden Markov Models
نویسنده
چکیده
Maximum Entropy (maxent) models are an attractive formalism for statistical models of many types and have been used for a number of purposes, including language modeling (Rosenfeld 1994), part of speech tagging (Ratnaparkhi 1996), prepositional phrase attachment (Ratnaparkhi 1998), sentence breaking (Reynar and Ratnaparkhi 1997) and parsing (Ratnaparkhi 1997). Maxent models allow the combination of many different types of information in a principled fashion.
منابع مشابه
Reduction of Maximum Entropy Models to Hidden Markov Models, Extended Version
We show that maximum entropy models can be modeled with certain kinds of Hidden Markov Models (HMMs). This allows us to easily construct maximum entropy-style models with hidden variables, hidden state sequences, or other characteristics. The resulting models can be easily trained using standard algorithms with guaranteed locally, and in some cases globally, optimal parameter settings. We also ...
متن کاملTaylor Expansion for the Entropy Rate of Hidden Markov Chains
We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...
متن کاملIntroducing Busy Customer Portfolio Using Hidden Markov Model
Due to the effective role of Markov models in customer relationship management (CRM), there is a lack of comprehensive literature review which contains all related literatures. In this paper the focus is on academic databases to find all the articles that had been published in 2011 and earlier. One hundred articles were identified and reviewed to find direct relevance for applying Markov models...
متن کاملChinese Chunking Based on Maximum Entropy Markov Models
This paper presents a new Chinese chunking method based on maximum entropy Markov models. We firstly present two types of Chinese chunking specifications and data sets, based on which the chunking models are applied. Then we describe the hidden Markov chunking model and maximum entropy chunking model. Based on our analysis of the two models, we propose a maximum entropy Markov chunking model th...
متن کاملMaximum-likelihood estimation for hidden Markov models
Hidden Markov models assume a sequence of random variables to be conditionally independent given a sequence of state variables which forms a Markov chain. Maximum-likelihood estimation for these models can be performed using the EM algorithm. In this paper the consistency of a sequence of maximum-likelihood estimators is proved. Also, the conclusion of the Shannon-McMillan-Breiman theorem on en...
متن کامل